RANSAC is an important algorithm in robust optimization and a centralbuilding block for many computer vision applications. In recent years,traditionally hand-crafted pipelines have been replaced by deep learningpipelines, which can be trained in an end-to-end fashion. However, RANSAC hasso far not been used as part of such deep learning pipelines, because itshypothesis selection procedure is non-differentiable. In this work, we presenttwo different ways to overcome this limitation. The most promising approach isinspired by reinforcement learning, namely to replace the deterministichypothesis selection by a probabilistic selection for which we can derive theexpected loss w.r.t. to all learnable parameters. We call this approach DSAC,the differentiable counterpart of RANSAC. We apply DSAC to the problem ofcamera localization, where deep learning has so far failed to improve ontraditional approaches. We demonstrate that by directly minimizing the expectedloss of the output camera poses, robustly estimated by RANSAC, we achieve anincrease in accuracy. In the future, any deep learning pipeline can use DSAC asa robust optimization component.
展开▼
